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CLAIMS 

1. An image processing method for estimating an output 
image from a plurality of input images, comprising the steps 
of: 

5 detecting a positional relationship among the plurality 

of input images captured by image capture means for 
capturing an image each pixel of which has n pixel values; 
and 

estimating the output image by identifying observed 
10 pixel components that should be taken into consideration 

from the plurality of input images for each pixel position 
of the output image, and producing the output image having 
(n + 1) or more pixel values for each pixel based on the 
identified observed pixel components. 
15 2. An image processing method according to claim 1, wherein 
in the step of estimating the output image, observed pixel 
components that should be taken into consideration are 
selected from the plurality of input images for each pixel 
position of the output image, and the output image having (n 
20 + 1) or more pixel values for each pixel is produced based 
on the selected observed pixel components. 

3. An image processing method according to claim 1, wherein 
in the step of estimating the output image, the output image 
having (n + 1) or more color signals for each pixel is 
25 produced from the plurality of input images, based on the 
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positional relationship detected in the detection step. 

4. An image processing method according to claim 1, wherein 
in the step of estimating the output image, the output image 
is determined taking into account a color correlation at 

5 each pixel position and/or in the vicinity of each pixel 
position of the output image. 

5. An image processing method according to claim 1, wherein 
in the step of estimating the output image, pixel values of 
pixels of the input images are changed to pixel values 

10 depending on the degree of steepness of an edge, and the 
output image is estimated based on the resultant changed 
pixel values . 

6. An image processing method according to claim 1, wherein 

n is equal to 1; and 
15 in the step of estimating the output image, the output 

image having three pixel values for each pixel is produced 
from the plurality of input images, based on the positional 
relationship detected in the detection step. 

7. An image processing method according to claim 1, wherein 
20 the image capture means is a single-plane sensor, in which a 

color signal observed varies depending on the position of 
each input image . 

8. An image processing method according to claim 1, wherein 

the image capture means captures the plurality of input 
25 images with underexposure; and 
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the image processing method further comprises the step 
of correcting the exposure by gaining up each pixel value of 
the plurality of input images. 

9. An image processing method according to claim 8, wherein 
5 the step of estimating the output image comprises the steps 

of: 

determining a corrected position of each pixel 
depending on the positional relationship detected in the 
detection step for each pixel of each of the plurality of 
10 input images; 

producing a light intensity integral constraint formula 
indicating that the difference between the integral of light 
intensity in a region centered at the corrected position and 
a pixel value of a pixel at the corrected position should be 
15 less than a predetermined error; and 

calculating the light intensity integral constraint 
formula. 

10. An image processing method according to claim 9, 
wherein when the light intensity integral constraint formula 

20 is produced, if the plurality of input images have been 

gained up, the error is changed depending on the factor of 
the gaining up. 

11. An image processing method according to claim 8, 
wherein in the step of estimating the output image, 

25 a corrected position of each pixel is determined 
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depending on the positional relationship detected in the 
detection step for each pixel of each of the plurality of 
input images; and 

the output image is estimated taking into account a 
5 pixel value of a pixel of the input images whose corrected 
position is located in the vicinity of each pixel position 
of the output image and also taking into account the error 
depending on the factor of the gaining up. 

12. An image processing method according to claim 1, 
10 wherein in the step of estimating the output image, 

a corrected position of each pixel is determined 
depending on the positional relationship detected in the 
detection step for each pixel of each of the plurality of 
input images ; 

15 the output image is estimated taking into account a 

pixel value of a pixel of the input images whose corrected 
position is located in the vicinity of each pixel position 
of the output image and also taking into account the 
distance between the corrected position and the position of 

2 0 the corresponding pixel of the output image. 

13. An image processing method according to claim 1, 
wherein in the step of estimating the output image, 

a corrected position of each pixel is determined 
depending on the positional relationship detected in the 
25 detection step for each pixel of each of the plurality of 



- 278 - 

S04P0831 

input images; and 

the output image is estimated taking into account a 
pixel value of a pixel of the input images whose corrected 
position is located in the vicinity of each pixel position 
5 of the output image and also taking into account the error 
depending on the factor of the gaining up. 

14. An image processing method according to claim 1, 
wherein in the step of estimating the output image, 

a corrected position of each pixel is determined 
10 depending on the positional relationship detected in the 
detection step for each pixel of each of the plurality of 
input images; and 

the output image is estimated by determining the 
weighted sum of pixel values of the input images whose 
15 corrected position is located in the vicinity of each pixel 
position of the output image. 

15. An image processing method according to claim 14, 
wherein in the step of estimating the output image, the 
correct pixel value of a pixel of interest is estimated by 

20 determining the weighted sum of pixel values of the input 
images whose corrected position is located in the vicinity 
of each pixel position of the output image, weights of the 
weighted sum being determined using a function having a 
lowpass-f ilter characteristic with respect to the distance 

25 between the position of the pixel of interest in the output 
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image and the corrected position. 

16. An image processing method according to claim 15, 
wherein in the step of estimating the output image, the 
correct pixel value of the pixel of interest is estimated by 
5 selectively performing 

a first calculation process of determining the weighted 
sum of pixel values of the input images whose corrected 
position is located in the vicinity of the position of the 
pixel of interest; or 

10 a second calculation process of determining the 

weighted sum of pixel values of the input images whose 
corrected position is located in the vicinity of the 
position of a pixel of the output image located adjacent to 
the pixel of interest. 

15 17. An image processing method according to claim 16, 
wherein in the step of estimating the output image, 

the second calculation process is selected when the sum 
of weights in the first calculation process is equal or 
nearly equal to 0 . 

20 18. An image processing method according to claim 15, 
wherein in the step of estimating the output image, the 
correct pixel value of the pixel of interest is estimated by 
selectively performing 

a first calculation process of determining the weighted 

25 sum of pixel values of the input images whose corrected 
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position is located in a first vicinity of the position of 
the pixel of interest; or 

a second calculation process of determining the 
weighted sum of pixel values of the input images whose 
5 corrected position is located in a second vicinity of the 

position of the pixel of interest, the second vicinity being 
greater than the first vicinity. 

19. An image processing method according to claim 18, 
wherein in the step of estimating the output image, the 

10 second calculation process is selected when the sum of 

weights in the first calculation process is equal or nearly 
equal to 0 . 

20. An image processing method according to claim 15, 
wherein in the step of estimating the output image, the 

15 correct pixel value of the pixel of interest is estimated by 
selectively performing 

a first calculation process of determining the sum of 
pixel values weighted by a first weighting factor of the 
input images whose corrected position is located in the 

20 vicinity of the position of the pixel of interest; or 

a second calculation process of determining the sum of 
pixel values weighted by a second weighting factor of the 
input images whose corrected position is located in the 
vicinity of the position of the pixel of interest. 

25 21. An image processing method according to claim 20, 
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wherein in the step of estimating the output image, 

the second calculation process is selected when the sum 

of weights in the first calculation process is equal or 

nearly equal to 0. 
5 22. An image processing method according to claim 1, 

wherein in the detection step, 

the positional relationship among the plurality of 

input images is detected with reference to an input image 

captured at the exact or nearly exact center of a time 
10 during which the plurality of input images were captured. 

23. An image processing apparatus for estimating an output 
image from a plurality of input images, comprising: 

image capture means for capturing the plurality of 
input images having n pixel values for each pixel; 

15 detection means for detecting a positional relationship 

among the plurality of input images; and 

means for estimating the output image by identifying 
observed pixel components that should be taken into 
consideration from the plurality of input images for each 

2 0 pixel position of the output image, and producing the output 
image having (n + 1) or more pixel values for each pixel 
based on the identified observed pixel components. 

24 . A program for causing a computer to execute a process 
of estimating an output image from a plurality of input 

25 images, the process comprising the steps of: 
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detecting a positional relationship among the plurality 
of input images captured by image capture means for 
capturing an image each pixel of which has n pixel values; 
and 

estimating the output image by identifying observed 
pixel components that should be taken into consideration 
from the plurality of input images for each pixel position 
of the output image, and producing the output image having 
(n + 1) or more pixel values for each pixel based on the 
identified observed pixel components. 



